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Teachers, administrators, and a few parents and 
students rated 100 teacher ccmpetency- statements with the help of the 
Besearch and Development Center for Teacher Education. These 
competency statements then served as the basis for the Professional 
Personnel Evaluation System (PPES) , which also consists of rating 
scales to fce completed by administrators, parents and students: 
detailed observation procedures: a handbook; and a computerized 
record of ratinas for school-by-school comparisons. After the first 
year of operation (1976-79) , participants felt that the new system 
was supe-iOT to the old method of teacher • t^'stin^. There was great 
concern, however, over the variability in teacher ratings by 
administrators across schools.' In 1979-BO other events intervened. 
5+aff time costs were too high tc iustify the new system, raising the 
speculatich that teacher testing may be the only way to establish a 
minimum competency level. tCP) 
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Austin experiences in and concerns about teacher testing have 
already been well documented for AREA participants (Holley 1978, 1979), 
Suffice it to say here that we tried teachtt testing in a limited way and 
at the direction of our school board considered the possibility of testing 
not only teacher recruits, but all teachers in our district. We concluded 
that the inherent problems were prohibitive. Among the difficulties we 
perceived were: 

. the high cost of developing valid tests that would withstand 
legal scrutiny, 

. the narrowness of function that could be measured adequately 
with paper and pencil tests, and 

, the impact testing would likely have on, recruitment efforts. 
Our district thus decided to explore an alternative taethod to assure teacher 
competence.- That alternative was the development of a new competency-based 
teacher evaluation system. 

In this paper I would like to describe the development of this 
new system, describe the system, and then come back to give some evaluation 
of how well it appears to be accomplishing the initial goals we set for it. 

Devalopment 

At the direction of the Austin Independent School District (AISD) Board 
of Trustees, the Department of Staff Personnel with technical support from the 
Office of Research and Evaluation (ORE) initiated development of a new personnel 
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evaluation system in 1977. The goals set for the system were fairly elaborate. 
We wanted to develop a system which: 

. Measured high-priority competencies. 

. Yielded data which would be more reliable and valid than 
past data. 

. Had as its fundamental purpose the Improvement of teaching 
performance and, therefore, provided a fruitful data base 
for assessing, providing, and evaluating staff development. 
. Would be feasible in terms of time and cost. 

Wo'ild provide a usable source of research data on teacher 
performance and its relation to student performance. 
Given these goals and the fact that we were given somewhat less than a year to 
have a new system ready for adoption, we set as our first priority the establish- 
ment of a set of competency statements ► 

The Office of Research and Evaluation with the help of the Research 
and Development Center for Teacher Education at the University of Texas conducted 
a survey of the literature to identify a list of competency statements that had 
already been researched. This search and subsequent validitation activities 
were supported in part through a cooperative project the district had underway 
with the R and D Center which was funded by the National Institute of Education. 
Although the research on competencies was limited and not of high quality, we 
did establish a long list of competency statements. We also added competency 
statements based on our knowledge of the current research on teacher effectiveness. 
Then the list was reviewed, condensed, and pared until we had approximately 
one hundred competency statements. 

In late summer of 1977 ORE, again with the help of the R and D Center, 
conducted a complex validation survey on these competencies. This survey 



has also been w^l documented both through district publications and through 
an AERA prsentatlon (Chrlstner. Malltz. Kugle. and Calkins. 1979). Essentially, 
the survey used matrix sampling to assure that every teacher had Input on 
the competencies selected for final Inclusion In the evaluation system. The 
competencies were also rated by principals. Instructional specialists, a 
sample of students, a sample of parents. antNall faculty In the University of 
Texas College of Educatlo^. This data remains), by the way. as a rich source 
of data as yet untapped about the value that different groups place on different 
aspects of teaching. The teacher sampling was done in such a way that comparisons 
are possible between teachers of low and high socio-economic-status students, 
teachers of different subject areas, grade levels, and so forth. The Research 
and Development Center has also been involved in a follow-up study that we hope 
will offer some validation of the competencies against teacher performance as 
measured by external observers and against student achievement. 

While the survey was being conducted. Department of Staff Personnel 
members were meeting with ORE staff to finalize other aspects of the system. 
Numerous decisions had to be made and once made turned into written material 
to documant the system. 

In addition, the politics of the situation had to, be attended to. 
Meetings were held with constituent groups affected by the future system to keep 
them informed and to provide opportunities for their input into decisions. 
Among these groups were the teacher associations, the administrator association, 
parent groups. Board committees, and interested individuals. 

The entire system was completed and went to the Board of Trustees in 
tt^e early months of 1978. It was approved. This was noteworthy because prior 
to this effort, a committee had worked on a system for two years and the Board 
had rejected their proposal three times in a row. The final rejection had led 
initially to the appointment of our two departments to the task. 



The Ptofeaslonal Personnel Evaluation System 

This was the name given to the new system iir recognition of the fact 
that It covered such personnel as counselors and librarians as well as 
teachers. It consisted of the following elements. 

63 Competency Statements, Out of the survey, sixty-three competency 
statements were finally chosen. Further reduction seemed Inadvisable because 
the ratings on these sixty-three were so uniformly high. The statements served 
as the core of the new system. 

Rating form. The statements were placed on a standard type of 
rating form which principals or their designates would use to record final 
ratings. 

Detailed proaeduree. Among the new requirements of the system 
were those requiring evaluators to conduct at least two observations of term 
contract teachers and thtee observations of new teachers* instructional 
coordinators to do evaluation observations and provide written feedback, and 
evaluators to have evaluation conferences with those being evaluated. Detailed 
procedures to follow when teachers were in contractual difficulty were spelled 
out. Recommended procedures and forms for conducting observations and develop- 
ing plans for competency improvement were also included. 

Parent and student input. A form was provided whereby parents could 
provide input regarding teacher performance, l-his input was voluntary in nature. 
Strong parent pressure in favor of student ratings of teacher performance led 
to the inclusion of a research project to gather this type of data in four schools. 
(This project was later dropped because of a Texas Attorney General's ruling 
that data gathered in such a project would be considered open-records data 
available to the public. This would have led to constraints on the research that 
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.oade it seem inadvisable.) In addition to that project, however, all teachers 
were urged to gather student input for their ovn use. Student input forms which 
could be used at different grade levels were made available to teachers. 

A handbook. Complete details of the system were put into a handbook 
so that they would be readily available. This handbook included sections on 
district policies and the required and recommended procedures, of course, but in 
addition other sections were designed to serve as a resource to the evaluator. 
One section gave a list of betiavioral descriptors for each competency. This sec- 
tion has gone through more revisions than any other part of the handbook. It 
attempts to give evaluators a good base for making rating decisions. Behaviors 
that should be present for a "3" or a "S" rating are differentiated. There is a 
list of data sources that should be considered in making decisions on rating a 
competency. There is a listing of the available training resources in the 
district and at our regional state service center related to each competency. 
There is a similar listing of resources related to the competencies needed by 
the evaluator in doing evaluation. Finally, there is a sourcebook of data- 
gathering instruments that can be used, ranging from observation 'instruments for 
special purposes to forms for student and parent input. 

A feedback plan. It was anticipated that feedback on ratings 
being given by Individual evaluators as compared to ratings given by others 
and by the district in general would be a powerful motivator toward improved 
rating reliability. Therefore, plans were made to place ratings on computer 
and to provide school-by-school data summaries. An example of a school profile 
is shown on the following pages. In addition to its main purpose, this data 
was Intended to assist schools in preparing campus staff development plans. 
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Figure la: School Profile, Teacher Evaluating Ratings 
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Figure lbs School Suroroary Teacher Evaluation Ratings 
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Figure lb: Continued 
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A training program for evatmtors. Training of evaluators was 
agreed to be the primary essential necessary to attaining the goals set for 
the system. This training was, in fact, seen as far more Important in getting 
valid ratings and in promoting the improvement of teaching practice than any 
other aspect of the system. Therefore, ela^rate training activities and 
^chedtiles were developed. The first wave of training, conducted in the spring 
of 1978, was designed to be sure that all evaluators knew and understood the 
new system. Figure 2 on the the following page illustrates charts that 

I 

were maintained, by ORE to show who was receiving training. This information 
has been used to structure make-up training durir.r; the 1978-79 school year. 
The second wave of training addressed the practical needs of evaluators 
in terms of their evaluation skills. Figure 3 describes the nature of some 
of this training. At some points, administrator testing was used to assure 
that the necessary information was being communicated and received. 

Implementation and Evaluation 

The new system went into effect in the fall of the 1978-79 school 
year. The year began with orientation sessions held first with total faculty 
groups and then with individuals up for ev^J^.uation during that year. 

The Office of Research and Evaluation observed in all training sessions 
for the year, conducted evaluation activities on those sessions, and collected 
survey and interview data from evaluators and evaluatees during the entire 
year to monitor and evaluate implementation of the new system. This effort 
was probably the largest scale, most intensive Implementation effort ever 
conducted by the school district. There was a relative lack of other interfering 
events in the 1978-79 school year that made this possible; that is, there were 
no major new federal or state programs, no new court orders, and no major 
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Figure 2: 

EVALUATOR TRAINING 
RECORD. 
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{tar<>h, ApHt 1978. Oritoc*elon awioxa h«id for »d»ijiiitr«tors 
In eh* Pro^MalonAl Ptrtotmtl Evmluaeion Syset*. 

May 10, 1978. In eh« G««»r*l PTlnclpali »t«e4ai, wxtl ••p«ce« of 
«v»lu«tlon lysen v*f dUcufsad ladvidiot tint oaiugwtoe, tb« 
tol* of eh* lMtruceion«i cootdiaaeor *td th« unilorti dtflaleloo of 
i »r r»tlot. AddielooUly, »mU ifoup tMrelats w«tt coodtjcwd 
eo t«i««t* b.h*Tlor dMCTiptori .e eh. "3," or good/««p*ce«d l.v.1 
for MCh of eh* $3 e««eh«r conp«e«nci«s. 

Jtcit, 1978. Kt eh« ••condary * vtd«ot«p*d l«iioa u««d 

M th« bails for obs«rr«elon«i craioins Inaudlog wrlelai up -.hi ob««r- 
i«cion«. AC eh« •iaa«»t*ry UfU «» •vaiuacion eat* study vai coa- 
ducead uaini a vldap tapad iaaaon for obMrvational erainiai a«id othar 
dau aourcaa and culalaacing la tha coaplatloo of an avaluation 
loatruMOt baaad on tha data gacharad. 

AuauMtt 1978, Savaral sasaloas of tha staff davalopaint workshop 
tor adalalatrators focuaad on cha avaluation syataa, Thaaa lodudad 
4 eanaral ovarrtaw of cha cacoaaandad avaiuatlon ptocsduras, « 
thraa-part sarlaa on a sacondary evaluation casa study and two saasions 
on eonfaraadns eaehnlquat and skills. 

StPttribw, 1978, Savaral alni-sastiona on tha avaluation systas waca 
of farad «a oakaups and for staff n«w to USD. In sddltloo to toplas 
alraady covarad, a oaw saaslon on cha coapatancy iaprovaaane plan 
was hald. 

Octpbtv. 1978, Evaluation training alai-«asslona for tha spaelal 
•raas adislnlstxaeors vara hald. Tha topics covarad vara obsatvation 
tralnlng» an avaluation easa study, confarancing and usa of cha 
coapatancy iaprova«snt plan. 

/aitum, 1^^^^ ^ oriantation sasaion on tha avaluation sysea* was 
hald for non-public school administrators of schools whars an AISD 
staff «attbar is aoployad. 

Hbrimy* 1^^9- * CTalniag sassion was conductad for sacbodary 
principals and inattuctional coordinators to ijsptova co«Buaicatioo 
channals saong avaluation esaa oaabars aspacially in tarms of 
obsarvaeiona and thair writs-ups.' 

Sunmtr, 1979 to Spring, 1979. In addition to tha tratniag offarad by 
AISD, tha Education Satvlca Cantar. Ragion xm has davalopad * wo 
day training sassion on tha Instructional supacvtsian procaaa. mt*** 
waialag aatsiona sttasa cha iisportanoa of bahaviorally daflaing 
araas of coaearn and working positivaly with tha profasslanal toward 
ix^tovaoant through dau gathariag, analysis and confarancing. 



Figure 3{ SYflOPSlS OF TMINING CONDUCTED FOR 
ADMINISTRATORS IN THE PROFESSIONAL 
PERSONNEL EVALUATION SYSTEM. 
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. adttittlstratlve chaages to interfere. Aa you will see in a moment, this was 
probably a rare year in a large urban district. At any rate, implementation 
level of the new program of teacher evaluation was very high. 

The first year evaluation of the new system's implementation was, 
in general, very positive. The system appeared to be well-accepted. The 
• new system received higher ratings of adequacy than the old .system had tn.baae- 
line data that had beeen collected. The figure below indicates also that those 
evaluated felt the system was helping them to improve. 



tun 

SiCOHDAtX 

cxrc&xsMSzo 

SECOWMit 
omtS HOT 

01 3caoou 




lOX ^ ZOf 4055 50 j 40X 70X 

ptKcitAcg WHO fnatm imwovbcjit 



SOX 



Figure 4: fmCWt Of PtOFISSIONALS SIJimY^J^ PERCBITO 
^ IMPROVBMKNT AS A REST3LT OF THE STSTEM. 



The average number of formal observations reiported by professionals 
was 3.64 to 6 . 35, indicating that administrators conducted well over the 
required number of observations. The chart at the top of the following 

.-A 

page indicates who was conducting observations Conferences were also 
held more often than they were required. 
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Figure T^- NUMBIE OF OBSERVATIONS REPORTED BY PROFESSIONALS 
OP FOR EVALUATION. 



^ The big question was, of course, were the ratings any more 
consistent across the district? Was there any evidence that they might 
be more reliable? The answer to the first question was obviously that great 
variability was still evident across the district. The two school profiles 
on the next pages show quite clearly that ratings varied a great deal from 
school to school. This variability was, of course, expected since the profiles 
were anticipated as a key to getting that variability reduced. With respect 
to reliability, we had anticipated that a first step in increasing reliability 
and validity of ratings would be the forcing of a normal curve distribution 
in the ratings. Therefore, principals had been told repeatedly that most of 
their ratings would fall in the "3," labeled "good," category while there 
would be about three percent of the ratings at each extreme. The emphasis was 
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Figure 5a: School Profile with Low Variability on Teacher Evaluation Ratings 
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Figure 5b: School Profile with Fair Variability on Teacher Evaluation Ratings 
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by competency and not by Individual. That is, an individual migbt have 
several high ratings and several low ratings with the majority of competency 
ratings falling in the "3" category. The effect of this emphasis was a^ general 
iSWering of the competencies as shown in the ratings for the new system versus 
those bein^ given on the old system. The desired variability was not present. 
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Figure MEAN EATINOS FOR THE 
1977-78 TEACHUl 
EVALUATIOBI FOBMS. 



Figure 7. MEAN MTWS ON THE 19?§*79 

PROFESSIONAL CHECiCLIST AND THl 
X978-79 TEACHER EVALUATION FORM. 



All in aU, the findings at Che end of the first year indicated that 
there was considerable promise of success for the system. Then, we entered 
the second year of implementation. The expectation for this year was that ^ 
a revision in the administrative evaluation system that would parallel and 
support the teacher evaluation system was to occur. Further training where 
needs were indicated yim to continue. The number of competencies had been 
reduced from 63 to 46 in response to general concern about the length of the 
evaluation form. Tti* was done by using correlation data from the spring 1979 
ratings. It was at this point that two events occurred that have interrupted 

16. 23 



* • progreas on the ayatetn. The diatrict'a ten year old deaegregation auit 

finally came to trial in late suianer and a declaion toward the end of the 
year mandated innediate January implementation of a large-scale buaing plan 



junior and aenior high buaing plans. It Was only after the federal circuit court 
and Interveners learned to their surprise that the Austin schools were on a 
quarter system which ended in March rather than a semester system that 
ended in January that ii^ementation of the plan has been postponed until the 
fall of 1980. In addition to the chaos that you can imagine all this has 
created, our superintendent announced in NOvetaber of 1979 that he would not 
seek a renewal of his contract which expires in the summer of 1980 becauae of 
a lack of support by the Board. These two events have usurped time and atten- 
tion to such an extent that no one has had time to devote real attention to 
the teacher evaluation process. 

It did n'^t come as a great surprise to us therefore that the new 
teacher ratings given to new employees late In the fall show some indica- 
tions of an upturn in average rating and no indication of improvement in 
variability. In addition, it has been necessary to put up quite a fight to 
have the evaluation data to plan staff development used rather than pursuing 
a big needs survey to set up extensive district staff development to support 
the desegregation efforts. This Indicated to us that the goal of having this 
data serve as a basis for staff development planning and ilellvery has not 



can be made about whether the approach the district chose as a way to assure 
teacher competence is viable. Indeed, whether or not it wiU be possible to 
regain "the momentum lost due to the chaotic events of the year probably depends 



' at elementary level and fairly extensive chan|ls in the already-existing 
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to . conalderable extent on the direction and strength of the new superintendent 

when he is finally chosen. 

In retrospect, I still find that the goals established for the system 
initially, although high, have just minimally justified the cost of the system. 
This cost is to he calculated primarily in terms of the teacher and administrator 
time required to Implement it. Even though it seems that the requirements for 
observation and con.^erenclng are minimal. X am keenly aware of 'how much time they 
consume. As ^/e in schools become even more aware of the centrality of time to the 
learning process for students, we also become jealous of that time for staff. 
Perhaps evaluators are naturally skeptical, but as I have watched the ratings come 
In, I also become increasingly doubtful that administrators can be led to making 
the hard decisions that are associated with identifying competency inadequacy. 
Therefore,' in the back of my mind. I carry the suspicion that if we are to 
establish a minimum competency level for personnel in our schools, the only wccy 
may be through examinations prior to employment or perhaps even prior to entry 
into training as an educator. 
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